restrict rest workers based on open files limit by dtrawins · Pull Request #4108 · openvinotoolkit/model_server

dtrawins · 2026-04-01T23:09:46Z

🛠 Summary

Default number of REST workers based only on the number of CPUs might exceed the number of open files limit. It can happen for example in Kubernetes with Xeon CPU with almost 400 CPU cores and ulimit 1024 files.
This change make the default number of REST workers safe in such conditions. It will include open files limit as criteria to allow initialization and open connections. The ration between rest worker could assumes open files for drongon, mediapipe graphs and client connections.
When the number of REST workers is set explicitly, and required open files number exceeds the limit, a clear error will be raised and OVMS will exit without attempt to load the model.

🧪 Checklist

Unit tests added.
The documentation updated.
Change follows security best practices.
``

Copilot

Pull request overview

Updates OVMS REST worker configuration to account for Linux open-files limits, preventing overly large rest_workers values and adding regression tests.

Changes:

Compute a Linux-specific default for rest_workers based on RLIMIT_NOFILE and add a validation cap tied to open-files availability.
Add death tests validating behavior when the open-files limit is low.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File	Description
`src/config.cpp`	Adds Linux open-files-based default/validation logic for `rest_workers`.
`src/test/ovmsconfig_test.cpp`	Adds tests that manipulate RLIMIT_NOFILE to validate the new `rest_workers` constraints.

Copilot · 2026-04-01T23:13:59Z

src/config.cpp

+// We need to minimize the number of default drogon workers since this value is set for both: unary and streaming (making it always double)
+// on linux, restrict also based on the max allowed number of open files
+#ifdef __linux__
+#include <sys/resource.h>
+const uint64_t MAX_OPEN_FILES = []() {


#include <sys/resource.h> is placed inside namespace ovms (and mid-file). Including system headers inside a namespace can unintentionally put all their declarations/types into that namespace and create subtle lookup/type issues. Move this include up with the other includes at file scope (outside the namespace) and keep only the constants inside ovms.

src/config.cpp

Copilot · 2026-04-01T23:14:00Z

src/config.cpp

+    if (restWorkers() > (MAX_OPEN_FILES - RESERVED_OPEN_FILES) / 5) {
+        std::cerr << "rest_workers count cannot be larger than " << (MAX_OPEN_FILES - RESERVED_OPEN_FILES) / 5 << " due to open files limit. Current open files limit: " << MAX_OPEN_FILES << std::endl;
+        return false;
+    }


On Linux, this check subtracts RESERVED_OPEN_FILES from MAX_OPEN_FILES without guarding MAX_OPEN_FILES <= RESERVED_OPEN_FILES. With unsigned types this can underflow and effectively disable the limit check. Add a guard (or use a saturating subtraction) before computing (MAX_OPEN_FILES - RESERVED_OPEN_FILES) / 5, and consider emitting a clear error when the open-files limit is too low to run REST workers at all.

Suggested change

if (restWorkers() > (MAX_OPEN_FILES - RESERVED_OPEN_FILES) / 5) {

std::cerr << "rest_workers count cannot be larger than " << (MAX_OPEN_FILES - RESERVED_OPEN_FILES) / 5 << " due to open files limit. Current open files limit: " << MAX_OPEN_FILES << std::endl;

return false;

}

if (MAX_OPEN_FILES <= RESERVED_OPEN_FILES) {

std::cerr << "Open files limit (" << MAX_OPEN_FILES

<< ") is too low to run REST workers. Increase the RLIMIT_NOFILE soft limit "

<< "or adjust the server configuration." << std::endl;

return false;

}

const uint64_t maxRestWorkersByFiles = (MAX_OPEN_FILES - RESERVED_OPEN_FILES) / 5;

if (restWorkers() > maxRestWorkersByFiles) {

std::cerr << "rest_workers count cannot be larger than " << maxRestWorkersByFiles

<< " due to open files limit. Current open files limit: " << MAX_OPEN_FILES << std::endl;

return false;

}

Copilot · 2026-04-01T23:14:00Z

src/test/ovmsconfig_test.cpp

 #include <regex>
+#include <sys/resource.h>


<sys/resource.h> (and the new getrlimit/setrlimit-based tests) are Linux-specific, but the include is unconditional. This will break builds on non-Linux platforms. Wrap the include and these two new tests in #ifdef __linux__ (or equivalent platform guard).

Suggested change

#include <regex>

#include <sys/resource.h>

#include <regex>

#ifdef __linux__

#include <sys/resource.h>

#endif

Copilot · 2026-04-01T23:14:00Z

src/test/ovmsconfig_test.cpp

+TEST_F(OvmsConfigDeathTest, restWorkersDefaultReducedForOpenFilesLimit) {
+    // limit allowed number of open files to 1024 to make sure that rest_workers count is too large for the limit based on number of cpu cores alone
+    int cpu_cores = ovms::getCoreCount();
+    struct rlimit limit;
+    ASSERT_EQ(getrlimit(RLIMIT_NOFILE, &limit), 0);
+    struct rlimit newLimit = {static_cast<rlim_t>(cpu_cores * 5), limit.rlim_max};
+    std::cout << "Setting open files limit to " << newLimit.rlim_cur << " to test that default rest_workers count is reduced based on open files limit" << std::endl;


This test comment says the open-files limit is set to 1024, but the code actually sets it to cpu_cores * 5. Please update the comment to match the behavior (or adjust the limit if 1024 is intended).

Copilot · 2026-04-01T23:14:00Z

src/test/ovmsconfig_test.cpp

+    int cpu_cores = ovms::getCoreCount();
+    struct rlimit limit;
+    ASSERT_EQ(getrlimit(RLIMIT_NOFILE, &limit), 0);
+    struct rlimit newLimit = {static_cast<rlim_t>(cpu_cores * 5), limit.rlim_max};
+    std::cout << "Setting open files limit to " << newLimit.rlim_cur << " to test that default rest_workers count is reduced based on open files limit" << std::endl;
+    ASSERT_EQ(setrlimit(RLIMIT_NOFILE, &newLimit), 0);
+


setrlimit(RLIMIT_NOFILE, ...) can fail if newLimit.rlim_cur exceeds limit.rlim_max (or if the process lacks permission to change limits). As written, cpu_cores * 5 might be > rlim_max on some systems/containers, making this test flaky. Consider clamping rlim_cur to limit.rlim_max and/or skipping the test with a clear message when the limit cannot be adjusted.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

dkalinowski · 2026-04-03T13:35:37Z

src/config.cpp

        return false;
    }
+#ifdef __linux__
+    if (restWorkers() > (getMaxOpenFilesLimit() - RESERVED_OPEN_FILES) / 6) {


why is it 6 here, but 7 in getDefaultRestWorkers()?

dkalinowski · 2026-04-03T13:36:12Z

src/config.cpp

+    if (maxOpenFiles <= RESERVED_OPEN_FILES) {
+        return static_cast<uint64_t>(2);  // minimum functional number
+    }
+    return std::min(static_cast<uint64_t>(AVAILABLE_CORES), (maxOpenFiles - RESERVED_OPEN_FILES) / 7);  // 5x rest_workers to initialize ovms and 2x rest_workers for new connections


can you add log that number of workers has been decreased due to limit of open files?

dkalinowski · 2026-04-03T13:39:29Z

src/test/ovmsconfig_test.cpp

+    char* n_argv[] = {"ovms", "--config_path", "/path1", "--rest_port", "8080", "--port", "8081"};
+    int arg_count = 7;
+    ovms::Config::instance().parse(arg_count, n_argv);
+    EXPECT_TRUE(ovms::Config::instance().validate());


this test doesnt check if number of workers has been changed

dkalinowski · 2026-04-03T13:40:36Z

src/config.cpp

        return false;
    }
+#ifdef __linux__
+    if (restWorkers() > (getMaxOpenFilesLimit() - RESERVED_OPEN_FILES) / 6) {


is default rest workers also max possible rest workers?

restrict rest workers based on open files limit

58ceadb

Copilot AI review requested due to automatic review settings April 1, 2026 23:09

Copilot started reviewing on behalf of dtrawins April 1, 2026 23:10 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

dtrawins and others added 4 commits April 2, 2026 09:18

style

d55e2b3

win build

5821c8a

Apply suggestions from code review

56c9b8d

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Merge branch 'main' into limit-workers

d1f8e70

dtrawins requested review from atobiszei and dkalinowski April 2, 2026 11:31

dtrawins added 5 commits April 2, 2026 16:38

fix tests

c452b51

fix win tests

502a853

fix win tests2

9d534d8

improve limits

eab2fea

style

f3af316

dkalinowski reviewed Apr 3, 2026

View reviewed changes

unit test

d83678c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

restrict rest workers based on open files limit#4108

restrict rest workers based on open files limit#4108
dtrawins wants to merge 11 commits intomainfrom
limit-workers

dtrawins commented Apr 1, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

dkalinowski Apr 3, 2026

Uh oh!

dkalinowski Apr 3, 2026

Uh oh!

dkalinowski Apr 3, 2026

Uh oh!

dkalinowski Apr 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

-    if (restWorkers() > (MAX_OPEN_FILES - RESERVED_OPEN_FILES) / 5) {
-        std::cerr << "rest_workers count cannot be larger than " << (MAX_OPEN_FILES - RESERVED_OPEN_FILES) / 5 << " due to open files limit. Current open files limit: " << MAX_OPEN_FILES << std::endl;
-        return false;
-    }
+    if (MAX_OPEN_FILES <= RESERVED_OPEN_FILES) {
+        std::cerr << "Open files limit (" << MAX_OPEN_FILES
+                  << ") is too low to run REST workers. Increase the RLIMIT_NOFILE soft limit "
+                  << "or adjust the server configuration." << std::endl;
+        return false;
+    }
+    const uint64_t maxRestWorkersByFiles = (MAX_OPEN_FILES - RESERVED_OPEN_FILES) / 5;
+    if (restWorkers() > maxRestWorkersByFiles) {
+        std::cerr << "rest_workers count cannot be larger than " << maxRestWorkersByFiles
+                  << " due to open files limit. Current open files limit: " << MAX_OPEN_FILES << std::endl;
+        return false;
+    }

Conversation

dtrawins commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛠 Summary

🧪 Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

dkalinowski Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

dkalinowski Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

dkalinowski Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

dkalinowski Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dtrawins commented Apr 1, 2026 •

edited

Loading